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Abstract 

We consider the design of coding schemes for the wireless two-way relaying channel when there is no channel 
state information at the transmitter. In the spirit of the compute and forward paradigm, we present a multilevel coding 
scheme that permits computation (or, decoding) of a class of functions at the relay. The function to be computed 
(or, decoded) is then chosen depending on the channel realization. We define such a class of functions which can 
be decoded at the relay using the proposed coding scheme and derive rates that are universally achievable over 
a set of channel gains when this class of functions is used at the relay. We develop our framework with general 
modulation formats in mind, but numerical results are presented for the case where each node transmits using the 
QPSK constellation. Numerical results with QPSK show that the flexibility afforded by our proposed scheme results 
in substantially higher rates than those achievable by always using a fixed function or by adapting the function at the 
relay but coding over GF(4). 

Index Terms 

Network coding, multilevel coding, two-way relaying, compute-and-forward 

I. Introduction 

Physical layer network coding (PLNC) or Compute and Forward is a new paradigm in wireless networks where 
each relay in a network decodes a function of the transmitted messages and broadcasts the value of this function 
to the other nodes in the network. This has been shown to provide significant increase in achievable rates for some 
networking problems Q], Q, Q. For a recent and approachable tutorial/survey of the key ideas behind PLNC 
with reliable decoding, we refer readers to 0]. For another broad tutorial/survey of PLNC results with an eye to 
practical implementation, we refer readers to 0. 

This work was supported by the National Science Foundation under Grant CCF 0729210. Parts of this work have been published at the 201 1 
IEEE International Symposium on Information Theory. 
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An example of such a problem where compute and forward has been shown to be effective is the two-way relaying 
system shown in Fig. [T] Here, node A has data to send to node B and vice versa. The relay R is included to assist 
in this communication, and it is assumed that there is no direct link between nodes A and B. Near optimal coding 
schemes have been designed to maximize the exchange rate for the case where there is no fading in the channel in 
ED, El, 10. Building on results from [7], these authors derive an upper bound on the capacity of | log(l + snr) 
and show that with lattice coding and lattice decoding a rate of ~ log(i +snr) is achievable. This problem has also 
been studied for case where there is fading in the channel, but each node perfectly knows the fading coefficients for 
each network link in O. It has been shown that near-optimal performance can be obtained at high signal-to-noise 
ratio (SNR) if each transmitter inverts its channel prior to transmission. The authors in [9| apply lattices with list 
decoding to the two way relaying problem with a direct link between nodes A and B. Finally, compute and forward 
schemes for multiple input multiple output channels have been considered in [10]. 

In this paper, the complex channel coefficients Ka and Kb are assumed to be perfectly estimated at each receiver 
but unknown to each transmitter. For this scenario, the authors in ifTTl introduce a scheme called denoise-and- 
forward which uses channel dependent denoising functions at the relay to minimize the symbol error probability. 
The relay chooses denoising functions so that the distance profile for constellation points with different labels is 
optimized. This improves the symbol error rate for transmissions between nodes A and B, however, denoising is 
performed purely at the symbol level. There is no natural extension to include error correction at the relay. 

Recently, a scheme called compute-and-forward, which allows both adaptation of decoding functions and error 
correction at the relay has been presented in fT2l . In this scheme, the relay decodes an integer combination of the 
transmitted codewords, where the integer combination is adapted according to the channel gains. They show that such 
a scheme can be implemented using nested lattice codes to take advantage of the duality between modulus arithmetic 
in prime order fields and the modular operations of lattice decoding. Their scheme requires the construction of infinite 
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dimensional lattice codes which is not practical. The results in [ 12 1 are extended in a remarkable way in 1131 . where 
an algebraic framework is provided to design lattices over principal ideal domains. However, their proposed coding 
scheme is also based on large dimensional lattice codes. 

In this paper, we propose a compute and forward scheme based on multilevel coding (MLC). Unlike the coding 
schemes in lfl2l . |[T3"1 . our proposed scheme does not result in a lattice code and uses only linear codes over small 
prime fields (for example, binary linear codes), and can therefore be implemented with lower encoding and decoding 
complexity. Yet, it facilitates error correction for a larger class of decoding functions than those proposed in lfl2l . 
This is because the class of functions for our scheme is derived from the large set of non-singular square matrices 
over F p in place of the set of non-zero elements in large prime order fields. To the best of our knowledge, such 
an idea of using multilevel coding and exploiting the linearity over the prime field to adaptively decode linear 
functions of transmitted codewords is new. Another important contribution in this paper is that our proof for the 
achievability of rates with the proposed multilevel coding scheme requires a non-trivial extension of the proof of 
achievability of rates for multilevel coding for the point to point case. 

This paper is organized as follows. The key elements of the problem are outlined in Section III] Our proposed 



solution is detailed in Section III An achievable rate for the proposed scheme during the MA stage is given in 
Section IV These rates are numerically determined for an example where nodes A and B transmit using a QPSK 
constellation in Section [V] Simulation results for a regular LDPC code are shown to corroborate the information 
theoretic results. Key results are reiterated in Section |VI) 

Throughout this paper we will use the following naming conventions. Vectors or sequences will be denoted by 
underlined variables such as x. Random variables will be denoted by upper case variables such as X, while their 
outcomes will be represented by lowercase variables. Matrices will be represented by capital boldface letters such as 
X. Subsets will be denoted by capital scripted letters such as X. If a variable is associated with a specific node, this 
will be indicated by a subscripted capital letter like x a- MLC sometimes requires us to split a data sequence into 
subsequences for parallel encoding and transmission over separate bit levels. Variables associated with a specific 
bit level will be indicated by a superscript like x k . A specific element of a vector or sequence will be referred to 
by an index in brackets like x[n]. 

II. Problem Description 

Each node in the relay network is assumed to be half-duplex, so communication is split into two stages, a multiple 
access (MA) stage and a broadcast (BC) stage. We assume perfect synchronization between the transmitters and 
mainly focus on the MA stage in this paper. 

A. Multiple Access Stage 

Nodes A and B each encode their binary messages u A and u B into codewords v A € Ca and v B € Cb where Ca 
and Cb are the codebooks used at the nodes A and B respectively. These codewords are mapped to sequences of 
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symbols s A ,s B £ Q N with \Q\ — 2 e . The relay receives noisy observations of the sum of these symbol sequences 
according to 

y R = h A s A + h B s B +w R (1) 

where h A and h B are complex fading coefficients, and w R is complex additive white Gaussian noise (AWGN). 
This induces an effective constellation Q B at the relay defined by 

Qr = {iR € C|gi? = h A q A + h B q B , q A , q B £ Q}. (2) 

B. Adaptive Decoding at the Relay 

The main idea proposed in this paper is the construction of a coding scheme such that the relay can reliably 
decode some function of v A and v B for a desired set of channel conditions H C C 2 . Specifically, we jointly design 
codes C A and C B and a set of decoding functions JF such that, for any (h A , h B ) £ W, there exists / £ T such 
that the relay can reliably decode f(v A) v B ) from y . We require that node A (B) must be able to unambiguously 
decode v B (v A ) from the output of f(v A ,v B ) with its knowledge of v A (v B ). For a given / e J, we will define 
an induced codebook at the relay as the codebook corresponding to / i.e. 

c f,R = {f(v A >v B )\v A £ C A , v b £ C B }- (3) 

It is important to understand the structure of Cf.n since the probability of error in decoding f(v A ,v B ) from y 
depends on h A , h B , and C/^. The main advantage of our proposed scheme is that it guarantees that choosing 
one codebook C A and C B at the transmitter can result in a good induced codebook Cf iR for a class of functions 
T. More specifically, it guarantees Cf t R is a member of the ensemble of random coset codes which is an optimal 
ensemble for achieving the uniform input information rate for the equivalent channel between f(v A ,v B ) and y R 
for all f £ J-. We restrict our attention to classes of functions T which are applied componentwise at the relay. 
The broadcast stage is fairly standard and is identical to that considered in Q, Q. 

III. Proposed Scheme 

A. Multilevel Encoder 

The system model for the multilevel encoder for nodes A and B and the channel model for the MA stage is 
shown in Fig. [2] The encoder at nodes A and B uses MLC with a different coset of the same linear code C used 
at each bit level. For a detailed description of MLC and achievable rates for the point to point channel see |[T4ll . 

The encoder is described as it pertains to node A to simplify notation. First, the message u A is split into 
sub-vectors u A , ...,u A which form rows of an I x K matrix 

U4= . (4) 
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Fig. 2. Block Diagram of MLC Coset Encoders for MA Stage. 



Each u A , {1, ...,£} is encoded with a linear code C with generator matrix G to get codewords , ■ ■■>T A - These 
codewords from the rows of an £ x N matrix 



(5) 



Finally, a random binary vector is added to each 7* . Each A^ can be thought of as coset leaders of a random 
coset of the original linear code. We obtain a codeword of a random coset given by v A = X A 
The random coset leaders form an £ x N matrix 



H A , ke {!,..,£}. 



(6) 



The resulting coset codewords v A form the rows of a binary £ x N matrix given by 



X A = U A G ®A A = 



(7) 



Thus each code C A , k £ {1, ■■■,£} will be a different coset of C. The k t h row v A of is then a codeword of 
C A . We use the two variables x A [n] and v A to refer to the nth column and kth row of X^ respectively because 
it will simplify our notation later. It should be mentioned here that much of the intuition about the main result in 
the paper is best obtained by ignoring the fact that cosets are used at each layer and simply considering the use 
of identical linear codes at each level in the MLC scheme. The coset matrix A^4 is included to symmetrize the 
effective channel at the relay (i.e. is necessary for the proofs to be correct). 
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Fig. 3. MLC address mapping example using QPSK. 



The nth binary address vector x A [n] G F| maps to a symbol s A [n] e Q through the use of a symbol mapping 
function M. : F 2 — >■ Q. An example of such a mapping function is given in Fig. [3] where Q is the QPSK constellation. 
As shown, the mapping function is usually derived by partitioning the set of signaling points in Q into equal sized 
subsets. Let S C {1, ...,£} be the subset of elements of x A which are fixed. Then we define the output of Ai with 
these input bits as a subset of points from Q according to 

M({x k A \k e S}) = { q e Q\ q = M{{x k A \k e S}, e S}), V e {0, l}}. (8) 

This means that the returned subset of constellation points is the subset whose address vectors are equal to the 
known bits for all indexes, S. The output of M{{x A \k £ S}) is 2 e ~' s i constellation points. 
In this example of Fig. [3] if S — {2} C {1, 2} and x\ = 1, then 

M({x k A \keS}) =M{U) ={M(01),M(ll)} = {-l,-i}. 

Here, \S\ — 1 and I = 2. Therefore M. returns 2 2_1 = 2 constellation points. 



B. Adaptive Decoding at the Relay 

As mentioned previously, the goal of the proposed scheme is to allow the relay to decode a function of the 
transmitted codewords. Similar to the compute and forward scheme, our scheme utilizes the linearity of the base 
code C and the fact that the relay knows and A^. If nodes A and B encode their messages as described, the 
set of decoding functions T which the relay can use for decoding is defined as follows. 

Define T> as the set of I x £ binary matrices which are invertible over F2. The set of functions we consider is 
given by 



T={f: F e 2 x F^ 2 -> V e 2 \f(x A ,x B ) = [D A D, 



D^DseO}. 



(9) 
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Therefore a given / € T is defined by some D^, e 2? from which the relay should attempt to decode a matrix 
X fyR given by 

r X A 



X flR = [T> A T> B } 

Due to the linearity of [Da,Db] and G, we can express the desired matrix Xf t R as 
Xf, R = [D A D B ] 



(10) 
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Here, we see that the matrix X/ ^ can be written in terms of an effective message TJ f.R and coset matrix A./,r 
which can be computed separately based on /. Thus the rows of Xf t R are codewords from a different coset code 
of C. Note that / is applied elementwise to the sequences s A and s B . 

\ 1 1 

For clarification, consider the case of i = 2. Let a function fi be defined by = Dg = . Writing 

1 

the vectors x A and and [x B x B ] T respectively, we see that Xf liR — [x 1 ^ R x ^ R ] T is given by 



10 10 
10 1 



This corresponds to the binary XOR function given by fi(x A ,x B ) = \x \ © 3)ri x\ ® x\ ^ 



Define another function f% using 



given by f 2 (x A ,x B ) = [x\ ®x 2 B ,x 2 A ® x B n 
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This is the rotated-XOR function 



Recall from Q, that v k [n] x k [n]. Thus using /i at the relay corresponds to decoding [^©w^] and [«^©?i|]. 
Similarly, applying f 2 at the relay corresponds to decoding [v A ® v B ] and [v* A ®v} B }. 

To illustrate the importance of choosing the decoding function / depending on (h A , h B ), consider an example with 
Q = -1, -j} = {M(00),M(01),M(11),M(10)} (i.e. QPSK with Gray Labeling). Further, let h A = e j6A 
and h B = &> Bb , and let 9 = 9 A — B be the phase difference. Consider the decoding functions 



fl( x Ai x b) ~ \ X A © X Bi X A © X B 
f2(x A ,X B ) = [x A ® X B , X A ® X B 
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Fig. 4. Effective constellation at relay for different values of 6. 

The resulting constellation Q R at the relay is shown for different values of 9 in Fig. [4] Note that the complex 
coordinates of the constellation points are exactly the same, but their labels are different based on 9 and / £ /b}- 
When 9 « 0, /i appears to have better performance than fa in terms of the distances between points with unequal 
labels. The situation is reversed when 9 ps |. This shows that the performance for a fixed decoding function can 
vary widely with 9 even when both \h A \ and \h B \ are large. 

As illustrated in Fig. |4j each / £ J 7 induces a mapping M.f,n between address vectors Xj R and constellation 
points q R £ Q R similar to M. for the point to point case. The relay is only interested in decoding Xf #, which will 
have t rows. Thus Ai forms a one-to-many map from length I binary address vectors to constellation points. 
Let S C {1, ...,£} be the subset of elements from x_f R which are fixed. Then, let X{ x k R \kes} — ^2 be the subset 
of Xf R 's with the same values for all points in S. For a given / £ T, the output of Ai f t u is 

■Mf, R ({x!} iR \k £ S}) ={q R £ Q R \q R = h A M(x A ) + h B M(x B ), f(x A ,x B ) £ X {x * R \ keS} }. (12) 

For the example in Fig. |4] Ai ;(11) would return the four constellation points labeled 11 in each figure. Ai /,#(1?) 
would return the eight constellation points in the union Aif ,#(11) U A^/,#(10). 

In order for nodes A and B to be able to unambiguously decode their desired messages, the authors in [ 1 1 1 show 
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that / must satisfy 



f(x A ,x B ) ^ f(x' A ,x B ) V x A ^ x A and x B 
f(x A ,x B ) ^ f(x A ,x' B ) Vijj/ij and x A . 



(13) 



We call functions that satisfy this property unambiguous. 
Lemma 1: For any T) Al T) B € T>, a decoding function 
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(14) 



is unambiguous. 

Proof: The proof follows from the invertibility of and D^. For some x A , suppose that there exists x B ^ x' B 
so that 
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which is a contradiction. 



IV. Achievable Information Rates 

A. Achievable Rate for a Given Function 

For a given / and fixed channel gains h A and h B the achievable rate region is given by the following theorem. 
This theorem is the key contribution of this paper. 

Theorem 1: Choose some fixed D^, D B e V and define 



x f p = f{x A ,x B ) = [T> A T> B ] 



(15) 



Choose a subset S C {1, ...,^} and define 5 = {1, ■■■,£} \«S. Divide S into p non-empty disjoint subsets Si, ...,S p 
so that Uf=i <5» = S- L et ^») * € {1, define p i.i.d. Bernoulli random variables with parameter \. At last, let 
each row of and X B be encoded using a different coset of the same linear code C. Then there exists a linear 
code C of rate 1Z for which the relay can reliably decode X/ ; a as long as 1Z satisfies 

n< _min l i{Y R -{x k fR \keS}\{x k fR \keS},{x k fR ®z i \keS i ] V*e{l,...,p}). (16) 

5,5,5i,...,5 p P J ' J ' J ' 
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For the special case when 1 = 2, the set of bounds described by ([16} are equivalent to 

K<mm{^I{Y R ;X\,X R ), I(Y R ;X R \X 2 R ), I(Y R ; X R \X R ), I(Y R ; X R , X R \X R © Z\,X R © Z\)}. (17) 
Note that 

I(Y R - X R , X%\X R ®Z 1 ,X 2 R ®Z 1 ) = I(Y R ; X R , X 2 R \X R © X 2 R ). 

That is, {X R © Z±,X R © Z x } and {X R © X R } carry the same information about X R and X R . 

Proof: The detailed proof is provided in the Appendix. However, the key steps in the proof are outlined below. 

Our proof uses the standard approach of deriving upper bounds on the probability of error for a joint typicality 
decoder averaged over a carefully chosen ensemble of codes. The ensemble considered here is the ensemble obtained 
by using random cosets of the same linear code for the different signaling levels in the multilevel coding scheme. 
The linear code is chosen from the ensemble of linear codes with randomly chosen entries in the generator matrix. 
The use of the same linear code in each level is an important ingredient in our proposed scheme since we allow 
the relay to freely take linear combinations of codewords from different signaling levels. However, this is also what 
complicates the proof. The ensemble used here is different from the often used ensemble of random coset codes 
used at each level in the multilevel coding scheme since the latter ensemble allows for independently chosen codes 
at each level. While the latter ensemble has been used widely to obtain achievable rates for MLC for the point to 
point channel and the multiple access channel, the former ensemble has not been analyzed in detail in the literature. 
The key contribution of our proof in the Appendix is to derive the achievable rates with the former ensemble with 
identical linear codes at each level. 

This can be accomplished since the use of the same linear code at each level ensures that for each / e J, 
Cf R , k € {1, ■■■,£} is a member of the ensemble used at the transmitters. The main complication that arises from 
this is that the pairwise independence assertion that is required in typical channel coding proofs [15| does not hold 
for certain classes of error events. Particularly, it is possible for the relay to correctly decode some rows of R 
while others may be in error. We handle this by splitting the union bound for error probability into separate classes 
of error events which are conditionally pairwise independent. 

The bound for the £ = 2 case can be derived by letting S, S, Si , 6>2 C {1,2} take the following values respectively. 



{S = 


{1,2},5 


= Mi 


= {1},5 2 = {2}} 


{S = 


{1},5 = 


{2}, 5! 


= {1}} 


{S = 


{2},5 = 




-{2}} 


{S = 


{1,2},S 


= 0,Si 


= {1,2}}. 



Notice that the first three terms in ( fTT) are also required by the proof for multilevel coding for the point to point 
channel. The last bound is a result of the requirement that each signaling level uses a coset of the same linear code. 
It would be required for the point to point case as well if the same codes were used at each level. ■ 
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It should be noted that the steps of the proof for theorem 1 can applied almost unaltered to the problem of 
finding the achievable rate for decode-and-forward if nodes A and B transmit using different cosets of the same 
linear codes at each level. In a decode-and-forward scheme, the relay attempts to reliably decode the messages 
transmitted from node A and B and then broadcasts a function of the received messages to nodes A and B. With 
a slight change to the channel model, the proof of theorem 1 can be applied to the problem of recovering the 2£ 
coset codewords which form the rows of 



X-AB — 



(19) 



Dividing the set {1,...,2£} into subsets S,S,Si, ...,S P as in theorem 1, we can show that X-ab can be reliably 
decoded as long as 7Z satisfies 

TZ< _min -I(Y R ;{X'X B \kGS}\{X'X B \keS},{X\ B ®Z i \keS i }Vie{l,...,p}). (20) 
S,S,S 1 ,...,S P p 

Therefore, by allowing the relay to choose between compute-and-forward and decode-and-forward, the maximum 
of the bounds given by (fT6l> and d20) is achievable. 



B. Universally Achievable Rate 

We say that a rate 7Z is universally achievable over the set % C C 2 if there exists a fixed linear code C of 
rate 7Z and coset matrices and A B such that for every (hA,hs) € T~L, the relay can reliably decode X/./j 
for some / e T . That is some X/.# can be decoded with arbitrarily small probability of error in the usual 
information-theoretic sense. The main result in this section is the following theorem. 

Theorem 2: For a fixed / E T and (Jia, h B ), define !Zf(hA, h B ) as the supremum of rates satisfying (jT6]» where 
Xj R = f(x A ,x B ), For any finite set of channel gains, H c C 2 , any rate 7Z such that 

7Z < min max lZf(h,A,h B ) (21) 
{h A ,h B )eu fe? 

is universally achievable. 

Proof: For a fixed finite WcC 2 and set of decoding functions T, define TV as the supremum of rates satisfying 
( |2"T| >. Define 5 > as the acceptable probability of error for a finite length code and choose a fixed 1Z < 7Z'. 

We will first consider an arbitrary (fiA,h B ) € H and / G T such that 1Z < 7Zf(hA,h,B)- Define £l N as the 
set of coset codes of the form {C, Aa, A^} which have length N. Thus, by increasing the value of N we form a 
sequence of ensembles of coset codes il. Define P(Err\VL N ) as the ensemble average probability of decoding error 
for the ensemble fl N , Define P(Err\C, Aa, As) as the probability of decoding error for a specific coset code. 

Define Q^ ad C tt N as 

N in a A r- nJV 



n» ad = {C, A A , A b e n N \P{Err\C, A A , A B ) > 5}. (22) 

and P(good\N) = l ^£f 



Then let = Sl N \ n^ ad . Define P{bad\N) = and P(good\N) = as the probability that a bad 
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or good code is selected uniformly at random from fl respectively. We know that 

P(Err\n N ) = P(bad\N)P(Err\n^ ad ) + P(good\N)P(Err\Sl» od ) 

> P(bad\N)S + P(good\N)P(Err\Slf ood ) 

> P(bad\N)5. 

The proof of theorem 1 relies on showing that lira P(Err\Q, N ) = 0. Therefore, there exists some Nq such that 

N—tco 

for any N > N Q , 

P(bad\N)6 < P{Err\n N ) < - =>■ P(bad\N) < - 

T T 

for some finite r > 2\H\. This means that < Note that choosing r > 2\7i\ is arbitrary but ensures that 

t will be "large enough" to complete the proof. 

We want to show the existence of some fixed {C, Aa, A^} G Ft N such that for every (Ha, he) G H there is 
some / G J- so that P(Srr|C, A^, As) < <5. We can apply the steps above to find a set Q^ ad (hA,hB) for every 
{fiAihs) G Since is finite the largest N required by any (/iaj^b) G "H must exist and be a finite integer 

Since r is chosen to be larger than 2\H\, the set 
must be non-empty because 

£ Kr(^Ml<l^ m i/2. 

(,h A ,h B )en 

Thus, since at least half of the codes are always good, there exists at least one coset code which allows reliable 
decoding for every [Ha, he) G % as long as 1Z < 1Z' . 

m 

Note that in order for this problem to be practically interesting, the set H should be meaningfully defined. It 
may seem more natural to evaluate our scheme based on the outage probability for a fixed transmission rate. We 
consider the universally achievable rate formulation for two reasons. First, the outage probability can be determined 
for the block fading channel using results from Theorems 1 and 2. Second, the universally achievable rate is useful 
for illustrating the flexibility of the proposed scheme to phase mismatch between nodes A and B. This is especially 
interesting if we consider a system where the relay is used to provide power control information to nodes A and 
B as in (8). 

V. Numerical Results 

A. Numerical Results for QPSK 

As an example, consider the case where nodes A and B transmit symbols from a QPSK constellation with Gray 
Labeling. Fig. [5] shows a plot of the achievable information rate tJZf^hAjhg) as given in ( fT7j ) for each function 
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Fig. 6. /(Yr; f{X A ,X B )) vs. for each / G T G fa- 

f E J- dependent on the phase difference 9 = 9a — 9 b for an SNR of 7 dB. H is the set of channel gains 

H = {(h A ,h B )\h A = e^,h B = e jdB } (23) 

where 9a, 9b G {0, 2ir} for a finite integer m. Thus |H| is finite but approximates the selection of any value 

of 9a and 8b arbitrarily closely. 

The dotted line indicates the universally achievable rate in bits per complex symbol for the proposed scheme 
which satisfies Theorem 2 for H. Note that different functions provide the best performance for different values of 
9 which reiterates the substantial benefit of decoding adaptively. Notice that a small increase in rate makes reliable 
decoding impossible for any f E J- for a significant range of 9; however, there are many (Ha, hs) $ H such that 
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3/ £ T for which reliable decoding is possible. 

B. Coding over GF(4) 

It is interesting to use this QPSK example to compare our MLC scheme to the case where nodes A and B encode 
their a data using a linear code Cgfa over F4 of rate 7Zgf4- The relay uses the set of decoding functions J-qfa 
corresponding to linear combinations of codewords in F4 of the form 

v R = f{v Al v B ) = av A ® fiv Bl a,/3eF 4 \{0}. (24) 

Node A can decode v B from v A and v R by 

v B =r 1 {av A )®v R . (25) 

Node B can recover v A similarly. The relay should be able to decode v R reliably as long as there exists some 
/ € Fgfa for which 

TZ GF4 <I(Y R ;f(X A ,X B )). (26) 

The value of I(Y R ; f(X A , X B )) for each possible / e J-qfa is plotted as a function of 9 in Fig. [6] with an SNR 
of 7 dB. Again the dotted line represents universally achievable rate for the H in ( |23] >. 

C. Comparison of Proposed Techniques 

These numerical results illustrate that the proposed MLC scheme facilitates better decoding flexibility at the relay 
than coding over F4 for this example. In fact, in an analysis of these functions based on the labeling of points 
in Qr, it can be seen that J-qfa C T. However, this improved flexibility comes at the cost of additional rate 
constraints on each / € T. The thick dashed line in Figs. [5] and [6] represents the rate which is achievable if the 
relay decodes using some / which is equivalent to the componentwise xor operation for multilevel coding or finite 
field addition for F4. The difference between these curves illustrates the effects of the additional rate constraints 
imposed by ([T6|. In Fig. [5] the last term I(Y R ;X R ,X% \X R ® Zi,X R ® Z\) in {FT) is dominant if 9 s» | for 
determining the achievable rate for this function. In Fig. [6] we see that this term does not need to be satisfied if 
nodes A and B use a linear code in F4. 

The universally achievable rate for the T~L in ( f2"3] > (i.e. the constant value given by the dotted line in Figs. [5] and [6]) 
is plotted as a function of SNR in Fig.|7]for the cases where the relay uses F or Tqf4- This value asymptotically 
approaches 1.5 bits per symbol for coding over F4. From Fig. [6] this appears to occur because Tqfa does not 
provide the relay with a decoding function which works well when ^ ~ f ■ This represents an extreme case, because 
the event \h A \ = \h B \ occurs with probability zero for many random fading processes. However, this illustrates 
that for PLNC it is possible for the universally achievable rate to be limited by specific (h A , h B ) G T~L even if each 
\h A \ and \h R \ is large. 
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Fig. 7. Universally achievable rates vs. SNR(dB) for proposed relaying techniques 




Fig. 8. Required SNR(dB) vs. 9 to reliably decode a rate | code 



D. Simulation Results 

To corroborate these theoretical results, we simulated the performance of a regular (3,6) low density parity check 
(LDPC) code. In Fig. [8] the required SNR for a rate \ code is plotted as a function of 9 = 9 a — 9 b for the case 
where (/lAi^s) G H for the H in ( |23] l. The solid curve represents the theoretically required SNR as determined 
by ( [TSy The dashed red curve represents the SNR for which zero bit errors occurred during 200 simulations of a 
length 10 5 code for each tested 9. For a point to point Gaussian channel using binary phase shift keying, it has 
been shown in [16| that the required SNR for a (3,6) LDPC code with iterative decoding is about ldB away from 
the Shannon limit for the same channel and modulation format. In Fig. [8] we see that this trend appears to hold 
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for our scheme as well. We leave more rigorous testing for future work. Note that to achieve the theoretical limit 
imposed by ( fT6| ) using structured codes, it will be necessary to design coding schemes which universally achieve 
the capacity for many channel conditions. It appears that the class of spatially coupled LDPC codes would be a 
good choice for this ifTTIl . 

VI. Concluding Remarks 
In this paper, we have proposed a coding scheme based on MLC for compute and forward or PLNC for the 
case when the channel is perfectly estimated at each receiver but unknown to each transmitter. We showed that 
MLC allows for decoding of a set of functions of the transmitted messages and the relay can choose one function 
from this set depending on the channel coefficients. In Theorem 1, we obtained an achievable rate for a fixed 
decoding function and channel realizations. In Theorem 2, we obtained a numerically computable expression for 
the universally achievable information rate over a set of channel realizations. Numerical results for QPSK suggest 
that the proposed scheme significantly outperforms the use of a fixed decoding function with binary linear codes 
and is better than using linear codes over F 4 . 

Appendix 
Proof of Theorem 1 

Theorem 1 states that for a fixed / 6 T, if each C\,...,C A and C B , ...,C B is a different coset of the same linear 
code C of rate 7Z, then there exists some C for which the relay can reliably decode X/.^ for a suitably chosen 7Z. 

A. Additional Notation 

A few definitions only necessary for this proof have been omitted from the main text but are included here for 
clarity. 

We refer to the noiseless observed sequence at the relay as 

q R = h A s A + h B s B . (27) 
Where q 6 Q R . Thus the relay observes the noisy observations 

y R = h A s A + h B s B +w R = q R + w R . (28) 

When it is necessary to refer to variables associated with different messages, we will refer to variables like Uj,A 
by the integer j E {0, 2 — 1}, whose binary expansion is given by node A's unparsed message u A . We assume 
that nodes A and B encode "Uj A .A and Vj B . B for transmission, and that the relay observes the noisy samples 
corresponding to X Ji? j R . Note that the index j R of the desired message is a function of j A , j B , and /. 

The relay will attempt to reliably decode ^-j R j,R from y R using a joint typicality decoder. Thus the decoder 
declares an error if either Xj B j tR is not jointly typical with y R or if some incorrect message X.j j R , j ^ j R is 
jointly typical with y . We derive an upper bound on the error probability for this decoder over the ensemble of 

— it 

random coset codes. Specifically, let the elements of G, A^, and As be i.i.d. Bernoulli random variables with 
parameter |. 
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B. Pairwise Independence of Codewords 

Here we provide a brief analysis of the ensemble of coset codes used by nodes A and B and observed by the 
relay. The following lemmas are stated as they pertain to a nameless encoder to simplify notation. Both lemmas 
appear as part of the proof of Gallager's Coding Theorem for Random Parity Check Codes [15 |. We include these 
proofs because the intuition behind some of the steps is used for other parts of the proof of Theorem 1 . 

Lemma 2: Let each element of G and A fc be i.i.d. Bernoulli random variables with parameter |. Then we have 



1 

2 

That is, the codeword Vj £ associated with message vector u k £ Ff can take any value with uniform probability 
over the ensemble of random coset codes. 

Proof: For a fixed G and w-, the output of the linear encoder j k = UaG must take some value in F^. Since 
X k can take any value with equal probability we have 

P{Y_] = v k ) = P(A k = v k ® j_ k ) = i V v k £ F». (30) 

■ 

Lemma 3: Let each element of G and X k be i.i.d. Bernoulli random variables with parameter i. Then for any 
j' 7^ j for which u k ^ u k ,, we have 



P(V k = ^) = — V G Ff . (29) 



1 



Vli,«-€^. (31) 



That is, the codewords v k and v k , associated with u k and u k , respectively are pairwise independent and uniformly 
distributed over F^. 

Proof: Suppose that u k and u k , differ in the mth position, and let g , i £ {1, .... K} refer to the i t h row of 
G. Then for any set of rows 

£l> ■■■'9_m-V9-m+V ■—>S-K 

there is some g m which gives v k v k , — -f k © 7™, any fixed value. By the construction of G and Lemma 2, g m 
and v k can take any value with uniform probability. We can conclude that 

P(Vj =Vj, V k =v k \u k ^u k ,) 
= P(V k = v k \u k 4)P(V>], =v k \v k ,u k ^ u k ,) 
= P(A k = v k 7*14 £ 4)P(G m = v k , © v k \v k ,u k u k ) 

= ^^ = ^^^^'^2- (32) 



The key idea behind each proof is the same. In Lemma 2, we see that the uniform distribution of X k implies the 
uniform distribution of v k . In Lemma 3, we see that the uniform distribution of G implies the pairwise independence 
of codewords corresponding to distinct messages. 
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C. Distribution of Received Signal 

The last step before deriving upper bounds on the probability of decoding error is to derive the distribution of 
the signal received by the relay over the ensemble of codes. 

Lemma 4: Let T G f™ x2e be a matrix of rank to, and let x T [n] £ F™ be defined by 



x T [n] = T 



-3 A ,A I 



l -3sM 



For a fixed T, define the set Q Xt .r by 

Qx^m = Ur € Qb\ir = h A M(x A ) + h B M{x B ) for some x A ,x B e F2 such that x T [n] = T 
Then the distribution of q R [n] conditioned on a; T [n] is given by 



p {Q R [n] = qn\x T [n]) 



1 



V qR e Q x i n ],R- 



(33) 



}■ (34) 



(35) 



Proof: For a given ~Uj AtA and \Jj B:B , each v^ A A ,Vj B b> ^ e is uniformly distributed in F^ by 

Lemma 2. Thus over the ensemble of codes, each element of X J4 _ A and X jB .s is an i.i.d. Bernoulli random 
variable with parameter ~. By (27) , we have 



g R [n] = h A M(x jAA [n]) + h B M{x jBB [n]). 



(36) 



Thus for fixed channel gains, q R [n] is a bijective function of x- A [n] and x- B [n]. For a given x T [n] we have, 



p GK ja ,aM = x A ,X jB>B [n] = x B \x T [ 



R t J J — JA.^U J —J 

1 



2 2t- 



W x A ,x B such that x T [n] = T 



That is the distribution of 2£ i.i.d. Bernoulli random variables with parameter | conditioned on to linear combinations 
of these variables is uniform over the space of outcomes satisfying the linear constraints. The result follows because 
q R [n] is a function of the address vectors. ■ 
By ( |28) , the conditional distribution of y R [n] on {xj r ^ R [n]\k £ S} is given by 



1 



E 



1 



P(yR[n]\{xl JM [n]\keS}) = ¥I ^ ^ _ 

q R eM f , R ({^ R , f , R [n]\kes}) v/7nJ 
where M. /,_r is the mapping function at the relay induced by / and the channel conditions 



IvrW-qrI' 



(37) 



D. Analysis of Error Probability 

The relay uses a joint typicality decoder to decode Xj R j tR from y R . For some fixed e > 0, define as the 
set of (X-jj y R,y R ) pairs which satisfy the definition of joint typicality given in [18|. The set is referred to as 
the jointly typical set. Let the event Ej, j € {0, 2 Kl — 1} be the event (X-jj iR , y ) s .4;^. The probability of 
error given that the codeword corresponding to j R is observed by the relay can be expressed 

2 Ke -l 

P(Err\j R ) = P(E jR U \J Ej\j R ). (38) 

3=^3 R 
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Applying the union bound, we get 



P{Err\j R ) < P{E jR \j R ) + £ 



(39) 



Recall that y is the result of the relay observing the symbol sequence associated with message j R . Thus by the 



joint asymptotic equipartition property (AEP) we have that for any e > 0, 

P(E jR \j R )<e 



(40) 



for sufficiently large N. 

The proof of the channel coding theorem for the general discrete memory less channel in [18| relies on upper 
bounding P(Ej\j R ) using the joint AEP. This is not straightforward here because Xjjj?, j ^ j R and y are not 
independent with the same marginals for certain classes of error events. For example, if £ = 2, we could have 



U 



j,f,R 



U 



j R J,R< 



v 



3R,f,R 



(41) 
. Thus for 



for some e u € Ff \ {0} and e v € \ {0}. This means that f,R ^ Vj R ,f,R but ^J,R 
this class of error events "Kjj, R and 'X-j R ,f r are not pairwise independent. Note that this class of error events is 
handled by the proof of the coding theorem for the multiple access channel lfl4l . lfl"8l . and lfT9l . In the coding 
theorem proof for the multiple access channel, it is possible for the receiver to correctly decode a codeword from 
one transmitter while making an error in decoding the codeword from a second transmitter. This has the same effect 
as correctly decoding the codeword on one level of a multilevel encoder while making an error in decoding the 
codeword transmitted on the second level. 

Unfortunately, choosing to use a coset of the same linear codes at each bit level introduces a new class of error 
events of the form 



X 7:/./.' - X »'./. 



(42) 



For this class of error events, the columns of the error matrix *r © ~Kj R j t R must be in {[0 0] T , [1 1] T }. This 
is the key difference between our proof and the proofs for the general multiple access channel or point to point 
channel with multilevel coding. 

We can move forward by splitting the sum in ([39| into different events for which ~Kjj.R and ~Kj R j.R are 
conditionally pairwise independent. Define a set of p < £ disjoint subsets Si,...,S p C {1,. ..,£}. Let t m be the 
smallest element of S m , and define the sets T — {ti, ...,i p }, S = Si U ... U S p , and S = {1, ...,£} \ S. For each 
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set of subsets, define an index set Jsi,...,S p given by 







* 








A = ' 


j 











(43) 



. 4,,/,* • * ••- - s " . 

Here each message error vector, e ui € F|^\{0]- Vie {l,...,p} satisfies e„ V i ^ i'. For the sake of 

simplicity, we will complete the analysis of error probability for the case where 1 = 2, and then extend the results 
to a general t. 

Case {£ — 2}: If I = 2, the subsets in (|43]l can be written as 



J{i} -- 

J{1,2} " 
J{1}{2} = < j 



u. 



jj,R - ,,-.(. i ' 



U 



j,f,R = Uj R J lR 





j 










j 








j 


§u,l 






j 


£u,2 





These subsets are disjoint and cover each error event, so 

J {1} U J {2} U J {lt2} U J {1}{2} = {0, 2 2K - 1} \ { JR }. 
Therefore, the union bound on the probability of error for 1 — 2 can be written as 

P(Err\j R )<P(E jR \j R )+ £ P(E 3 \ jR , j e 

+ ^ PiEjljuje J {2} ) 

j£J{i,2) 

+ E p ( E i\jnJe ^{i}{2})- 

We define e„ j = e Mi G, i £ {1,2} as the codeword error vector associated with subset Si. The subscript u 
or v is used to differentiate between the message error vector and codeword error vector respectively. Over the 
ensemble of codes, each e v i is uniformly distributed in F^, and codeword error vectors e Vfi ,e V A, i 7^ j are pairwise 
independent and identically distributed. These facts can be shown using steps similar to the proofs of Lemmas 2 
and 3. 



(44) 



21 



If {j £ J{i}} i s gi yen > then we know that 



±11,1 



By Lemmas 2 and 3, wj n ^ ^ and v}j t R are pairwise independent and uniformly distributed on F^. Lemma 2 also 
tells us that v 2 h * R and v 2 j R are equal and uniformly distributed on F^. 

Define v R as the common value taken by * R = v 2 R j R . Joint AEP provides an asymptotically tight upper 
bound to each P(Ej\j R ,j £ J{i}) if we can show that 

P(X jtftR ,Y R \j R ,j £ J {1} ) = P(K j>f>R \j R ,j £ J {1} )P(Y R \j R ,j £ J {1} ). (45) 

This is equivalent to showing that 

P(X jJ>R ,Y R r%.j £ J {1} ) = P(X j!f , R \v 2 R J £ J {1} )P(Y R \v R ,j £ J {1} ). (46) 

for each value of v R . Therefore, consider some arbitrary fixed v R . We can use |28) and the definition of conditional 
probability to get 

PpLjj,R,Y R \v 2 R ,j£j {1} ) 

= /':X,,.„ rj..j £ J {1} )P(Y R \X^ R ,v 2 R ,j £ J {1} ) 

= P(K jJ<R \vl,j £ J{i})P(Q R + W R \v 1 J j R ,v 2 j R ,v R ,j £ J {1} ). (47) 
We see that, conditioned on v R , q R is a random function of Vj r j r , 

which is defined elementwise by Lemma 4. By Lemma 3, Vjf R and v^ r ^ R are pairwise independent, therefore 
j R and q R are independent. Since v 2 j R = v R we have 

P(Event\v 2 fR ,v 2 R ,j £ J {1} ) = P(Event\v 2 R ,j £ J {1} ) 

for any event. We conclude that 

P(Q R + W R \vl LR ,vl LR ,v 2 R ,j £ J {1} ) 
= P(Q R + W R \v 2 >LR ,v R ,jeJ {1} ) 

= P(Q R + W R \v 2 R ,j£j {1} ). (48) 
This allows us to conclude that |46) holds so we can use lfl8l Theorem 15.2.3] to get the following bound 

P{E \j Rl] £ J {1} ) < 2 -W*i*iin)-3*). (49) 
Similar steps can be used for the case when j £ 3ii\ to get 

P(Ej\j R ,j £ J {2} ) < 2-W*^l^)-3=). (50) 
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For the case when j € i7{i,2}> we have 



The most direct way to find a bound for this case is to reassign the address vectors so that this case is similar to 
the case when j e J{i}- Define a binary matrix D{i,2} given by 

1 
1 1 



D 



{1,2} 



(51) 



Then define a mapping function A4f tR (-) by 

■Mf, R {x fR [n}) = Mf, R (T> {h2} x fR [n}). (52) 
Then define effective codeword matrices X.j R j^ R and ~X-jj, R by 

X ,<-,M.' = l} : :.:.>! ././.' 

= D { i ;2 }X ji/ifl = X , Q 

This is the same as the case where j e if the relay observes the y R corresponding to codeword matricies, 
Xj R jji with the mapping function Mf^ R . Therefore for the case where j e J{i,2}> we have the bound 

P(Ej\j R ,j G J {1 , 2} ) < 2- N ^ Y ^\xl)-^) (53) 

which can be expressed in terms of the original address variables as 

P(Ej\j R ,j € J { i, 2} ) < 2- w «^I^H)- 3e ). (54) 

By the definition of mutual information, we have 

I{Y R -X R \X R ®X R ) 

= H(Y R \X R e X%) - H(Y R \X R ,X R e 

= e x£) - h(y r \x r ,x 2 r ) 

= I{Y R -X R ,X R \X R ®X R ). 
Therefore the bound is equivalent to 

P(EAm,3 e J { i, 2} ) < 2-^( / (^^-^i x -®^)- 3e ). (55) 

Lastly, for the case when j <G J{i}{2}, ^-j R j,R an d ~^j,f,R &K i.i.d. by Lemmas 2 and 3. We can therefore use 
joint AEP directly to get the bound 



P(Ej\j R ,j e ^ {1}{2} ) < 2- w (^^^«^)-3 £) _ 



(56) 
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Applying the upper bounds for each index set to ( |44| ), we get the following bound 

P(Err\j R )<e + ]T 2 - N ^ x k\x R )Se) 
jej {1} 

+ 2~ JV(/(Yj?;X « |x « ) ~ 3<s) 

+ V 2" Ar(/(yR;X «^« ) " 3e) . (57) 

There are 2 Nn — 1 elements in the sets Jsa\, J{2}> an d J{i,2}> an d there are fewer than 2 2Nn elements in the last 
set J7{i}{2}- Thus the upper bound on the probability of error for this code ensemble can be expressed 

P(Err\j R ) <e + 2 N ^- I ( Y n;X R \x R )+Se) 

+ 2 N(1l-I(Y R ;X 2 R \X 1 R )+3e) 

+ 2N(TZ-I(Y R ;X R ,X R \X R (BX R )+3e) 

+ 2 N(2K-I{Y R ;X 1 R ,X 2 R )+3e)_ (5g) 

Each of these terms can be made arbitrarily close to zero by increasing N as long as 1Z satisfies 

TZ < mzx(I(Y R - X R \X 2 R ),I(Y R ; X 2 R \X R ), I(Y R ; X R , X%\X R ® X 2 R ), i J(Y fl ; X R , X%)). (59) 

Note that this proof holds for an arbitrary j R which means that the bound holds independent of the transmitted 
message. 

Case {£ > 2}: For a general I, the proof is very similar. We split ([39) into the disjoint classes of error events in 
d43l to get 



P(Err\j R )<P(E jR \j R ) + J2 E E P ( E A3rJ e Js u ...,s p ). (60) 

p=i Si,...,s p jeJs 1 ,.-.,s p 

Then we find upper bounds on the probability of error for different classes of error events. 

First, we consider the case where each S m , m € {1, •••,£>} contains only its smallest element t m . This first case 
is analogous to the case where {j £ <J{i}} f° r the proof when I — 2. By Lemmas 2 and 3, we have 

' 2- 2N , keS 

2~ N , v x = v 2 and k £ S (61) 

j V.\ 7^ V-2 an d k ^ S. 

That is if k € S then v J j R and u* j r are independent and uniformly distributed. If k $ S they are equal and 
uniformly distributed. Let v R , k £ S be the common value taken by the k t h row of lLjj iR and Xj R f iR . 
The joint AEP gives an asymptotically tight upper bound to P(Ej\j R ,j £ Jsi,...,S ) if we can show that 

P(X jJtR ,Y R \j R ,j e Js u ...,s p ) = P(X jJiR \j R ,j e J S] s, )P(Y R \j R ,j e (62) 



p(X5, f ,R = = v 2 \j g j Sl _ Sp ) = 
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This is equivalent to showing that 

P(X jj!R ,Y R \{v k R ,k?S},jeJ Sl ,...,s p ) 

= P(X jJ<R \{v R ,k?S},jeJ Su ..., Sp )P(Y R \{v R ,ktS},jeJ Sl ,...js p ) (63) 

for each possible set of values {v R ,k g" S}. We can use ( |28] l and the definition of conditional probability to get 

P(X jJjR ,Y R \{v R , k £ S},j e J Sl ,..,s p ) 

= P(X ii/>fl |{^ , k S},j e J Sl ,..., Sp )P(Y R \X jJtR , {v R ,k? S},j e ^,..,5,) 
= P(X j!f , R \{v R ,k?S},j e Js u ...,s p )P(Q R + W R \X jtftR ,{v R ,k<?S},jeJ Sl ,..., Sp ). (64) 
Thus the problem simplifies to showing that 

P(Q R + w R \x^ R7 {v R ,k?S}j e J Sl ,...,s p ) 

= P(Q R + W R \{v k R ,k?S},j€j Sl ,...,s p ). (65) 

Since we are conditioning on {{v R , k g S}, j <G Jsi,...,S p }> the values taken by j R , k £ S are already given. 
Therefore we have 

P(Q R + W R \X jJ , R ,{v k R ,k?S},je J Su ...,s p ) 

= P(Q R + W R \{v^ R ,m&S},{v R ,ktS},jeJ Sl ,..., Sr ). 
The value taken by q R conditioned on {v R , k £ S} is a random function of {vj^ j R , m G S}, 

l R = 9({v? R j, R , meS};H, 

which is defined element wise by Lemma 4. Therefore by the independence of {v" l j R , m g S} and {wj^ j R , m E 
S} we can conclude that q R is conditionally independent of {v^f R , m £ S} given {{v R , k S}, j G J75 li ...,5 p }. 
Therefore since w fi is independent of any message, we can conclude that ( |6"5j ) holds. This is equivalent to (|62]> 
which allows us to apply joint AEP to get the upper bound 

i Jl; .j e J Sl ,...,s p ) < 2- N w^ x «' fe 0s})-3e)_ (66) 

To extend this result to the case where each S%, S p can contain multiple elements, we make this problem look 
like the first case. Define a matrix whose m t h column d m is given by 

d tk [n\ = t V fc = 1, 

I , n £ S k 

![ 77 = 777 
V m £ T. (67) 
, n^m 
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For example, if I = 6, S± = {2, 4, 5}, and 6>2 = {3, 6} we have 

1 



D 



{2, 4, 5}, {3, 6} 



1 

1 

10 10 

10 10 

1 1 

Define effective codeword matrices ~Kjj tR and X JH j >R by 

^-j,f,R = B Sl ,...,S p ^-jJ,R 



Then the k t h row u ■ * R of ~Kjj,R is given by 



v 



3R,f,R 



' J -3R-J,R® Zv,m ) 



= t™, 771 = 1, 



(68) 



(69) 



for some set of pairwise independent error vectors e v x , e v _ € \ {0}. 
For the £ = 6 example, this means that 



X 



£2,u 





This is the same as the case where each Si,...,S p contains only one element. Thus, we can apply the bound in 
(|66]l to get 

P(Ej\j R ,j S J Sl ...., Sp ) < 2- N ^ Y -^ ^r}|{^, ^r})-3 £ ). (70) 

The only step that remains is to show that the mutual information in f70| i can be expressed as 

I(Y R :{X k R , keT}\{X R , k?T}) 

= I(Y R ; {X R \k e S}\{X k R \k # S}, {X k R ®Z i \keS i }Vi= 1, ...,p) (71) 

where S — Uf = i <%> an d eacn %i i s an auxiliary random variable which is Bernoulli distributed with parameter ~. 
By d68j, we have 



X R ® x fi 



, keSuT 



(72) 



We therefore have 

{x k R \k eT}« {x k R \k g T} 

£ T} & {x k R \k £ S] U |J {x k R © x^|fc G 5 m \ {i m }}. 

m— 1 

The mutual information can therefore be expressed 

I(Y R ;{X k \keT}\{X k \k?T}) 

= I(Y R ; {X k \k G T}\{X k R \k $ S} U (J {X* ffi |* G 5 m \ {t m }}) 

m— 1 

= ff(y R |{X*|fc £ 5} U |J {X* © G 5 m \ {i m }}) - , 

m— 1 

The last equality follows because if we know x l R and x k R © x^jT then we know both x l R and x R . Which tells 
that knowing {x R \k eT}U {Jm=i{ x R © ^jf \k G <S TO \ {t m }} is equivalent to knowing {ir|.|fc G S}. 
It can be shown for each S m , m G {1, ... , p} that 

{x k R © x^" \k G <S m \ {t m }} © z m |fc G 5 m }. (7 

For example, if we consider our £ = 6 case, we have 6>i = {2, 4, 5}. If we know that 

(x R © x%, x R © = (a, 6) , a, 6 G {0, 1} 

then we have 

[x%x R ,x R ) £ {(0,a,b),{l,a,b)} 

which is equivalent to knowing 

{x R © Z\ , X R © Z\ , X R © 24 ) • 

We therefore have 

I(Y R ;{X k R \keT}\{X k R \k^T}) 

= H(Y R \{X k R \k ?S}u(j{X k R ® X% \k £ S m \ {t m }}) - H(Y R \X R , X R ) 

rn = 1 
P 

= H(Y R \{x k \k $ s} u |J {x k © z m \k g 5 m » - ff(y R |^, ...,X R ) 

= I(Y R ; {X k \k G 5}|{^|fc £ 5}, {X* © Z t \k G <Sj} V i = 1, (7 
This is the same as ( |7Tj ), which allows us to restate the bound in f70] i as 

< 2-JV(/(y /? ;{xJj|fee5}|{^|fc^5},{x*©z ! |fee5i} v i=i,..., P )-3£) 

A 2-A r (-f(^n;5i,...,5 p )-3e) ^ 
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The last step defines a mutual information I(Y R ;Si, ...,S p ). This slight abuse of notation simplifies the last few 
steps of the proof. 

Plugging this into d60l, we have 



P(Err\j R )<P(E jR \j R ) + J2 £ £ 2- Jv ^a^,...,5,)-3 e)< (76) 

p=lSi,...,S p jGj- Sl ,..., Sp 

For each possible 5i, ...,6> p we have 

1^,...^! - (2 NR - l)(2 NK - 2)...(2 NK ~p) < 2 N ^. 

Therefore we have 

P(Err\j R )<e + J2 £ 2 Nn ^2- N ^ Y ^-' S ^-^ 

p=l 5i,...,5 p 

I 

<e + £ 2 A'(Kp-7(y„;5 1 ,...,5 I ,)+3e)_ 
p=l Si,...,S p 

This bound approaches zero as long as 

K< _min -I(Y R] {X%\keS}\{X^\ke'S} ) {X^®Zi\keSi}\/i€{l,...,p}). (77) 

5,5A,...,5 P P 

This completes the proof. 
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